MMID: Multimodal Multi-view Integrated Database for Human Behavior Understanding
نویسندگان
چکیده
This paper introduces the Multimodal Multi-view Integrated Database (MMID), which holds human activities in presentation situations. MMID contains audio, video, human body motions, and transcripts, which are related to each other by their occurrence time. MMID accepts basic queries for the stored data. We can examine , by referring the retrieved data, how the different modalities are cooperatively and complementarily used in real situations. This examination over different situations is essential for understanding human behaviors , since they are heavily dependent on their contexts and personal characteristics. In this sense, MMID can serve as a basis for systematic or statistical analysis of those modalities, and it can be a good tool when we design an intelligent user interface system or a mul-timedia contents handling system. In this paper, we will present the database design and its possible applications .
منابع مشابه
Efficiency of Speech Recognition for Using Interface Design Environments by Novel Designers
Previous studies on usability of graphical design-widgets, like menus and buttons, proposed the use of speech and non-speech (earcons and auditory icons) for solving their usability problems. In this paper we investigate speech as an input metaphor to enhance learnability, or the ability to use a system with no prior knowledge, in order to design interfaces using a multimodal interface design t...
متن کاملA novel robust speech recognition algorithm based on multi-models and integrated decision method
In this paper, a new robust speech recognition algorithm of multimodels and integrated decision(MMID) is proposed. A parallel MMID(PMMID) algorithm is developed. By using this new algorithm the advantages of different models can be integrated into one system. This algorithm uses different acoustic models at the same time based on DDBHMM (duration distribution based Hidden Markov Model)[2]. Thes...
متن کاملA Brain Inspired Approach for Multi-View Patterns Identification
Biologically human brain processes information in both unimodal and multimodal approaches. In fact, information is progressively abstracted and seamlessly fused. Subsequently, the fusion of multimodal inputs allows a holistic understanding of a problem. The proliferation of technology has exponentially produced various sources of data, which could be likened to being the state of multimodality ...
متن کاملMultimodal human behavior analysis: Learning correlation and interaction across modalities Citation
Multimodal human behavior analysis is a challenging task due to the presence of complex nonlinear correlations and interactions across modalities. We present a novel approach to this problem based on Kernel Canonical Correlation Analysis (KCCA) and Multi-view Hidden Conditional Random Fields (MV-HCRF). Our approach uses a nonlinear kernel to map multimodal data to a high-dimensional feature spa...
متن کاملA Multi-view Hyperlexicon Resource for Speech and Language System Development
New generations of integrated multimodal speech and language systems with dictation, readback or talking face facilities require multiple sources of lexical information for development and evaluation. Recent developments in hyperlexicon development offer new perspectives for the development of such resources which are at the same time practically useful, computationally feasible, and theoretica...
متن کامل